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(54) VIRTUAL STORAGE CONTROL SYSTEM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To virtually supply a lot of 
memories to a built-in program to be operated in an 
electronic equipment from the outside through a 
network. 

SOLUTION: In this virtual storage control system, a 
node 12 incorporated with a CPU 121 for executing a 
built-in program 1 22 using a virtual storage is connected 
to a memory server 1 4 provided with an external storage 
device for storing contents of memory pages in the 
virtual storage, a page management mechanism 141 and 
a page save/return part 142 through a network. When a 
request for saving or returning a memory page in the 
virtual storage is generated from the built-in program 
(client) 122, the memory server 14 is informed of the 
request through the network 1 1 and the save or return 
processing of the requested memory page is executed 
by the page save/return part 142 in the server 14 by 
using the external storage device 1 40. 
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(57) [Abstract] 

[Object] To make it possible to virtually provide a large quantity of memory externally 
via a network, with respect to a built-in program operating in electronic equipment. 
[Means] The configuration is such that a node 12 having a built-in CPU 121 that 

1 0 executes a built-in program 1 22 using a virtual storage, and a memory server 1 4 
having an external storage unit 1 40 that sores the entity of a memory page of the 
virtual storage, a page management mechanism 141 , and a page saving/return 
section 142 are connected to each other by a network 11 , and when a saving or return 
request of the memory page in the virtual storage is made from the built-in program 

1 5 (client) 122, the request is notified to the memory server 14 via the network 11 , and the 
saving or retum processing of the requested memory page is performed by the page 
saving/return section 142, by using the external storage unit 140. 

[What is claimed is] 
20 [Claim 1 ] A virtual storage control system comprising: 

first equipment having a CPU built therein, which executes a built-in program, 
using a virtual storage; 

second equipment including a large capacity memory in which the entity of the 
memory page in the virtual storage is stored, and a page management unit that 
25 manages the respective memory pages; and 



a network to which the first and the second equipment are connected, 
wherein the first equipment comprises a page saving/retum request unit that 
transmits a saving or return request of a memory page to the second equipment via 
the network, when the saving or retum request of the memory page in the virtual 
5 storage is made from the built-in program operating in the equipment, and the second 
equipment comprises a page saving/return unit that performs the saving or return 
processing of the requested memory page by using the memory, in response to the 
request from the page saving/retum request unit in the first equipment. 
[Claim 2] The virtual storage control system according to claim 1 , wherein the 

1 0 first equipment is constructed so that a plurality of processes can be operated in 
parallel, and 

the page management unit in the second equipment manages the respective 
memory pages, together with identification information of the process using the page. 
[Claim 3] The virtual storage control system according to claim 2, wherein a 

1 5 plurality of the first equipment is connected to the network, and 

the page management unit in the second equipment manages the respective 
memory pages, together with identification information of the first equipment using the 
. page. 

[Claim 4] The virtual storage control system according to claim 1 or 2, wherein 

20 a plurality of the second equipment is connected to the network, and 

the first equipment further comprises a page saving destination/return source 
determination unit that notifies all the second equipment on the network of a request for 
a saving destination or a retum source of the memory page by broadcast, in order to 
determine the saving destination or the retum source of the memory page, and 
25 determines the saving destination or the return source of the memory page, based on 
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the response with respect to the notification. 

[Claim 5] The virtual storage control system according to claim 1 or 2, wherein 

a plurality of the second equipment is connected to the network, and third equipment 
that determines the equipment to be a saving destination or a return source of the 
5 memory page from the second equipment is also connected thereto, and 

the first equipment further comprises a page saving destination/return source 
determination unit that inquires of the third equipment via the network, when it is 
necessary to determine the saving destination or the return source of the memory 
page, and determines the saving destination or the return source of the memory page 
1 0 based on the response with respect to the enquiry. 

[0005] 

[Problems to be solved by the Invention] 

Recently, the function required for the built-in program is upgraded. 
1 5 Accompanying the up-grading, the program becomes more complicated, and hence, 
high maintainability of the program and easiness in reusing the program parts are 
required. 

[0006] Highly developed programs require a large quantity of memory, and a large 
quantity of memory and a highly developed memory management mechanism (virtual 
20 storage and the like) are also required in order to use a language, which can be 
described by a program having high maintainability and reusability (such as Java 
developed by Sun Microsystems in USA). 

[0007] However, conventional electronic equipment (built-in equipment) such as 
management equipment in which the built-in program operates has a strict restriction in 
25 the quantity of the memory that can be used, due to the production cost and the like. 
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Therefore, in this type of conventional electronic equipment, it is difficult to apply a 
highly developed built-in program. 

[0008] The present invention has been achieved in order to solve the above 
problems. It is an object of the present invention to provide a virtual storage system 
5 that can virtually supply a large quantity of memory to the built-in program operating in 
the electronic equipment externally via a network, thereby enabling application of the 
highly developed built-in program. 

[0017] 

1 0 [Embodiments of the Invention] 

The embodiments of the present invention will be explained with reference to 
the accompanying drawings. 
[0018] 

[First Embodiment] 

1 5 Fig. 1 is a block diagram illustrating the configuration of a virtual storage 

control system according to the first embodiment of the present invention. In this 
figure, electronic equipment (built-in equipment) having a built-in CPU 121 , for example, 
management equipment (hereinafter, node) 12 is connected to a high-speed network 
11 such as Ethernet, Asynchronous Transfer Mode (ATM), or Fibre Channel. Various 

20 objects to be managed (not shown) are connected to the node 12 via an input and 
output interface device 13. 

[001 9] In the node, the CPU 1 21 executes a built-in program 1 22 that uses a virtual 
storage (and comprises a built-in OS and a program operating under management of 
the OS). The node 12 has a built-in small-capacity physical memory (actual memory) 
25 123. In the node 1 2, a virtual memory 1 24 having a large capacity exists equivalently. 
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which can be seen from the built-in program 122. This virtual memory 124 is provided 
by a memory server 14 described below. The data of a memory page (actual page 
data) in the virtual memory 124, to be processed in the node 12, is an-anged in an 
optional area on the actual memory 123, so that it can be processed by the CPU 121 
5 in the node 12. 

[0020] To the network 11 is also connected a memory server (memory server 
equipment) 14 realized by information processing equipment such as a personal 
computer or a workstation. The memory server 14 is for providing a virtual memory 
124 (upper page) to the built-in program 122 on the node 12, and includes a 

1 0 large-capacity external storage unit 1 40 that stores the entity of the page on the virtual 
memory 124. The external storage unit 140 can be realized by a disk array using 
hard disks, as shown in the example of Fig. 1 , as well as by a semiconductor disk, a 
DVD-RAM and the like. The memory server 14 has a page management 
mechanism 141 that manages memory pages stored (saved) in the external storage 

15 unit 140, and memory pages provided to (return in) the built-in program 122 on the 
node 12, and a page saving/return section 142 that executes saving or return 
. processing of the memory page requested from the built-in program 122 in the node 
12, by using the external storage unit 140. 

[0021] The operation in the system having the configuration shown in Fig. 1 will be 
20 explained next. It is assumed here that saving and return of the page (memory page 
in the virtual storage) on the virtual memory 124 become necessary, by the operation 
of the built-in program 1 22 on the node 12. In the case of the general information 
processing equipment such as a personal computer applying the virtual storage 
method, since it has a large-capacity memory such as a hard disk or a semiconductor 
25 disk as a part of the equipment, saving and retum of the memory page is performed 
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with respect to the memory. However, since the node 12 as the built-in equipment in 
this embodiment does not have a large-capacity memory, it cannot perform saving and 
return of the memory page, as performed by the general information processing 
equipment. 

5 [0022] Therefore in this embodiment, the built-in program 122 operating in the node 
12 performs saving and return of the memory page by requesting it to the external 
memory server 14. In other words, the built-in program 122 has a function of 
requesting saving and return of the memory page. Therefore, hereinafter, the built-in 
program 122 in the node 12 is referred to as a client 122. 

1 0 [0023] When a memory page is for example saved, the client 1 22 side transmits the 
memory page information having the structure as shown in Fig. 2, that is, the memory 
page information including the contents 21 of the page to be saved (actual page data) 
21 and page management information 22 including an address (virtual address) VA of 
the page on the virtual memory 124, from the node 12 to the memory server 14 via the 

1 5 network 1 1 . The memory server 1 4 side stores the received memory page 

information in the external storage unit 140 by the page saving/retum section 142. 
Alternatively, only the actual page data 21 of the memory page information may be 
stored in the external storage unit 140, and the page management information 22 may 
be stored and managed by the page control mechanism 141 . 

20 [0024] When the memory page is returned, the client 1 22 side transmits a virtual 
address VA of the page to be returned from the node 12 to the memory server 14 via 
the network 11, to request transmission of the contents of the page. 
[0025] The memory server 1 4 specifies the memory page requested from the client 
11 by the page management mechanism 141 , based on the virtual address VA 

25 transmitted from the client 1 22. Then, the memory server 1 4 fetches the information 
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of the requested memory page from the extemal storage unit 140 by the page 
saving/return section 142, and transmits the information to the requested client 122 via 
the network 11 . The client 122 arranges the information of the requested memory 
page transmitted from the memory server 14 in an optional area on the actual memory 
5 1 23, so that the CPU 1 21 can process the information in the node 1 2. Thereby, the 
return processing of the memory page by the cooperation operation with the memory 
server 14 in the node 12 is finished. 

[0026] When the client 1 22 has a parallel execution function of a plurality of 
processes, the respective processes have the respective virtual memories (virtual 

1 0 spaces) virtual memory 1 24 on the node 1 2. In this case, the information of the 
memory page transferred between the memory ser\/er 14 and the client 122 may 
include page management information 32 in which information for identifying the 
process (process ID) is added, as shown in Fig. 3, instead of the page management 
information 22 in Fig. 2. 

15 [0027] 

[Second Embodiment] 

In the first embodiment, an example of a system in which one memory server 
and one client exist is shown. However, the present invention is also applicable to a 
systeni in which a plurality of memory servers and a plurality of clients exist. 

20 Therefore, the second embodiment, in which the present invention is applied to a 

virtual storage control system including a plurality of memory servers and a plurality of 
clients, will be explained with reference to the accompanying drawings. 
[0028] Fig. 4 is a block diagram illustrating the configuration of a virtual storage 
control system according to the second embodiment of the present invention. In this 

25 figure, a plurality of built-in equipment, for example, two built-in equipment (nodes) 42-1 
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and 42-2, and a plurality of memory servers, for example, two memory sen/ers 44-1 
and 44-2 are respectively connected to the network 41 (corresponding to the network 
11 in Fig. 1). 

[0029] The nodes 42-1 and 42-2 respectively include a CPU 421-1 , 421-2. a built-in 
5 program (client) 422-1 , 422-2, and a small-capacity actual memory 423-1 , 423-2 built 
therein. In the nodes 42-1 and 42-2, a virtual memory 424-1 , 424-2, which can be 
seen from the client 422-1 , 422-2, respectively exists equivalently for the number of 
processes that can be parallel executed by the clients 422-1 and 422-2. The virtual 
memory 424-1 , 424-2 is provided respectively by a memory server 44-1 , 44-2. 

1 0 [0030] In the system shown in Fig. 4, since a plurality of clients (here, the client 422-1 
in the node 42-1 , and the client 422-2 in the node 42-2) exists, the memory page 
information transferred between the memory server 44-1 , 44-2 and the client 422-1 , 
422-2 also includes information for identifying the node (client) (node ID). In the 
systeni of Fig. 4, since the respective processes have the respective virtual memories 

1 5 (virtual spaces) 424-1 and 424-2 on the nodes 42-1 and 42-2, information for 

identifying the process (process ID) is also required as in Fig. 3. In this embodiment, 
therefore, page management information 52 in which the node ID is added, as shown 
in Fig. 5, is used, instead of the page management information 32 shown in Fig. 3. 
[0031] In the system in Fig. 4, since a plurality of memory servers exist (here, two 

20 menriory servers, 44-1 and 44-2), the system requires a mechanism for determining 
which server becomes the memory page saving destination from a certain client. 
Two methods can be applied for the determination of the memory page saving 
destination, that is, a first saving destination determination method in which the 
memory server to be the saving destination is determined in advance one for each 

25 client, and a second saving destination determination method in which the memory 



server to be the saving destination is dynamically determined every time the memory 
page is saved. 

[0032] Here, the procedure of the first saving destination determination method will 
be explained, with reference to the sequence chart in Fig. 6, using an example in which 
5 a memory page saving destination firom the client 422-1 in the node 42-1 is 
determined. 

[0033] The client 422-1 notifies all memory servers on a network 41 simultaneously 
of a request for a memory server that becomes the saving destination of the memory 
page (step S1). This notification can be easily realized, by using the mechanism of 
1 0 broadcast, if the network 41 applies a protocol referred to as an Internet Protocol (IP), 
for example, in Ethernet. 

[0034] The respective memory servers 44-1 and 44-2 give a response 
conresponding to the respective situations (saving destination server notification or 
non-saving destination server notification) to the client 422-1 . Here, the memory 

1 5 server 44-2 returns a saving destination server notification notifying that the memory 
server 44-2 becomes the saving destination seiver for the memory page from the client 
422-2 (step S2), and the memory server 44-1 returns a non-saving destination server 
notification notifying that the memory server 44-1 does not become the saving 
destination server for the memory page from the client 422-2 (step S3). 

20 [0035] Upon reception of these notifications, the client 422-1 selects a memory 

server as the saving destination of the memory page. Here, the memory server 44-2 
is selected as the saving destination. Thereafter, the client 422-1 transmits the 
memory page information having the structure as shown in Fig. 5 from the node 42-1 
to the memory server 44-2, to request saving of the conresponding memory page, 

25 every time it becomes necessary to save the memory page. 



9 



[0036] In the above explanation, not only the memory server that becomes the 
saving destination of the memory page, but also the memory server that does not 
become the saving destination gives a response, but the configuration may be such 
that the memory server that does not become the saving destination does not give a 
5 response. 

[0037] The second saving destination determination method will be explained. In 
the second saving destination determination method, the request for the saving 
destination memory server by broadcast from the client 422-1 to the respective servers, 
which is applied in the first saving destination determination method, is transmitted 
1 0 every time the saving of the memory page becomes necessary, not in advance. 

Therefore, in the second saving destination determination method, the memory server 
that becomes the saving destination is determined every time the memory page is to 
be saved. 

[0038] As shown in Fig. 7, it is also possible to have a configuration such that an 
1 5 arbitrator 70, which is a special server having a function of determining a pair of a client 
that becomes a saving source (or a retum destination) of the memory page and a 
memory server that becomes a saving destination (or a return source), is added to the 
system in Fig. 4, so that the respective clients 422-1 and 422-2 inquire of the arbitrator 
70 about a memory server that becomes the saving destination (or the return source) 
20 of the memory page. 

[0039] The same method as in the first saving destination determination method, that 
is, the method in which the client requests the arbitrator 70 to determine the memory 
server to be the saving destination in advance, and the same method as the second 
saving destination determination method, that is, the method in which every time 
25 saving of the memory page becomes necessary, the client requests the arbitrator 70 to 
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dynamically determine the memory server to be the saving destination, can be applied 
to the system of Fig. 6. 

[0040] Return of the memory page in the system in Fig. 4 or Fig. 6 will be explained, 
using an example when the client 422-2 returns the memory page. 
5 [0041 ] At first, in the system in Fig. 4 or Fig. 6, since a plurality of memory servers 
exists, when the client 422-2 returns the memory page, it is necessary to determine 
from which server the memory page is to be returned, that is, determine the return 
source memory server, as in the instance when the memory server to be the saving 
destination is detemiined. Either one of methods can be applied for the determination 

10 of the retum source memory server of the memory page, that is, a first retum source 
determination method in which it is set such that return is always performed from the 
saving destination server, and a second return source determination method in which 
every time the return of the memory page becomes necessary, the memory server to 
be the return source is dynamically determined. 

1 5 [0042] The first return source determination method has the simplest processing, and 
the client 422-1 needs only to memorize the memory server that becomes the saving 
destination. When this method is applied, however, there is a problem in that a large 
load is applied on a specific memory server, according to the system situation. 
[0043] The second return source determination method is to solve the above 

20 problem in the first return source determination method, and it is based on the 

assumption that the memory page moves between memory servers according to the 
load status and the like, that is, the saving destination memory sen/er does not always 
agree with the return source memory server, in order to realize load balancing between 
the memory servers. 

25 [0044] When the second return source determination method is applied to the 

11 



system in Fig. 4, every time the return of the memory page becomes necessary, a 
request for the retum source memory server is transmitted by broadcast to all memory 
servers (44-1 , 44-2) on the network 41 , by the same method as the second saving 
destination determination method, to determine the retum source memory server 
5 based on the response thereto. 

[0045] When the second return source detennination method is applied to the 
system in Fig. 6, every tim6 the retum of the memory page becomes necessary, the 
client may inquire of the arbitrator 70 about the retum source memory server, and 
determine the return source memory server based on the response thereto. 

10 [0046] 

[Effects of the Invention] 

According to the present invention, in the built-in equipment, a large quantity of 
memory can be used by the virtual storage, without providing an external storage unit 
such as a hard disk. 

1 5 [Brief Description of the Drawings] 

[Fig. 1 ] A block diagram illustrating the configuration of a virtual storage control 
system according to the first embodiment of the present invention. 
[Fig. 2] A diagram illustrating an example of data structure of memory page 
information applied in the embodiment. 

20 [Fig. 3] A block diagram illustrating an example of data structure of memory page 
infonnation, when a client (built-in program) parallel-executes a plurality of processes. 
[Fig. 4] A block diagram illustrating the configuration of a virtual storage control 
system according to the second embodiment of the present invention. 
[Fig. 5] A diagram illustrating an example of data structure of memory page 

25 information applied in the embodiment. 
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[Fig. 6] A sequence chart for explaining the procedure of a first saving destination 
determination method applied in the embodiment, using an example in which a 
memory page saving destination from a client 422-1 in a node 42-1 is determined. 
[Fig. 7] A block diagram illustrating a modification example of the configuration shown 
in Fig. 4, 

[Description of Signs] 
11,41 - Network 

1 2, 42-1 , 42-2 - Node (built-in equipment, first equipment) 
14, 44-1 , 44-2 ■■• Memory server (second equipment) 
22, 32, 52 Page management information 
70 ■• Arbitrator (third equipment) 
121, 421-1, 421-2 - CPU 

122, 422-2, 422-2 ••• Built-in program (client, page saving/retum request unit, page 
saving destination/return source determination unit) 

123, 423-1 , 423-2 ■ • Actual memory 

124, 424-1 , 424-2 ••• Virtual memory 

141 Page management mechanism 

142 "■ Page saving/return section 
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